Under-display camera

ABSTRACT

An under-display camera is provided, wherein an image acquisition unit including the under-display camera is configured to generate an evaluation image based on a first light, by using an imaging unit including an image sensor. The first light is output by a light emitting unit, including a light source, and passes through a display panel. The under-display camera is controlled based on a result of evaluating a performance of the under-display camera, the result being obtained by calculating a feature value based on the evaluation image, calculating a reference value based on a reference image, and comparing the feature value with the reference value, the reference image being obtained based on a second light output by the light emitting unit, the second light not passing the display panel.

CROSS TO REFERENCE TO RELATED APPLICATION(S)

This application claims priority to Korean Patent Application No.10-2020-0189709 filed on Dec. 31, 2020 in the Korean IntellectualProperty Office, the disclosure of which is incorporated herein byreference in its entirety.

BACKGROUND

Example embodiments of the present disclosure relate to an under-displaycamera.

With the development of the mobile industry, display technology includedin mobile phones has also been developed. Current display technology hasbeen developed to increase a screen share of a display panel. Also, withrespect to a front camera of a mobile phone, a technique of increasing ascreen share of a display panel may have an excellent effect in terms ofproduct design, but there may be a limitation in that image quality maybe deteriorated. Particularly, with respect to deterioration of imagequality occurring in an under-display camera (UDC) which may increase anarea in which a display panel may be disposed, there is a difficulty inevaluating accurate UDC performance.

SUMMARY

One or more example embodiments provide an under-display camera (UDC)which may, by calculating a numerical feature value based on anevaluation image obtained by an imaging unit through a display panel,quantitatively evaluate performance of the UDC, and may have improvedperformance.

According to an aspect of an example embodiment, there is provided anunder-display camera configured to, when disposed below a display,generate an image based on a light passing through the display, whereinan image acquisition unit including the under-display camera isconfigured to generate an evaluation image based on a first light, byusing an imaging unit including an image sensor, the first light beingoutput by a light emitting unit, including a light source, and passingthrough a display panel, and wherein the under-display camera iscontrolled based on a result of evaluating a performance of theunder-display camera, the result being obtained by calculating a featurevalue based on the evaluation image, calculating a reference value basedon a reference image, and comparing the feature value with the referencevalue, the reference image being obtained based on a second light outputby the light emitting unit, the second light not passing the displaypanel.

According to an aspect of an example embodiment, there is provided anunder-display camera configured to, when disposed below a display,generate an image based on a light passing through the display, whereinan image acquisition unit including the under-display camera isconfigured to generate an evaluation image based on a first light, byusing an imaging unit including an image sensor, the first light beingoutput by a light emitting unit, including a light source, and passingthrough a display panel, and wherein the under-display camera iscontrolled based on a result of evaluating a performance of theunder-display camera, the result being obtained by converting theevaluation image to a contour image; calculating, in each of a pluralityof positions on an outline of the contour image, a feature value basedon a relative location of each position with respect to a center of thecontour image; and comparing the feature value with a reference valueobtained from a reference image, the reference image being obtainedbased on a second light not passing through the display panel.

According to an aspect of an example embodiment, there is provided anunder-display camera configured to, when disposed below a display,generate an image based on a light passing through the display, whereinan image acquisition unit including the under-display camera isconfigured to generate a first evaluation image and a second evaluationimage based on a first light, by using an imaging unit including animage sensor, the first light being output by a light emitting unit,including a light source, and passing through a display panel, andwherein the under-display camera is controlled based on a result ofevaluating a performance of the under-display camera, the result beingobtained by calculating a first feature value based on the firstevaluation image and calculating a second feature value based on thesecond evaluation image; and obtaining a first performance evaluationresult by comparing the first feature value with a reference valueobtained from a reference image and obtaining a second performanceevaluation result by comparing the second feature value with thereference value, the reference image being obtained based on a secondlight not passing through the display panel.

BRIEF DESCRIPTION OF DRAWINGS

The above and other aspects, features, and advantages of the disclosurewill be more apparent from the following detailed description of exampleembodiments taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 illustrates development of a camera provided with a displaypanel;

FIG. 2 is a block diagram illustrating a device for evaluatingperformance of an under-display camera according to an exampleembodiment;

FIG. 3 is a flowchart illustrating a method of evaluating performance ofan under-display camera according to an example embodiment;

FIGS. 4A and 4B illustrate evaluating performance of an under-displaycamera according to example embodiments;

FIG. 5 is a diagram illustrating a display panel for applying a methodof evaluating performance of an under-display camera according to anexample embodiment;

FIG. 6 is a flowchart illustrating a method of evaluating performance ofan under-display camera according to an example embodiment;

FIG. 7 is a flowchart illustrating a method of evaluating performance ofan under-display camera according to an example embodiment;

FIGS. 8 and 9 are diagrams illustrating images generated using a methodof evaluating performance of an under-display camera according toexample embodiments;

FIGS. 10 and 11 are diagrams illustrating a parameter used in analysisof an image with respect to a method of evaluating performance of anunder-display camera according to example embodiments;

FIGS. 12 and 13 are diagrams illustrating a method of calculating areference value and a feature value with respect to a method ofevaluating performance of an under-display camera according to exampleembodiments;

FIG. 14 is a diagram illustrating a performance evaluation result of amethod of evaluating performance of an under-display camera according toan example embodiment;

FIG. 15 is a flowchart illustrating a method of evaluating performanceof an under-display camera according to an example embodiment; and

FIG. 16 is a diagram illustrating a performance evaluation result of amethod of evaluating performance of an under-display camera according toan example embodiment.

DETAILED DESCRIPTION

Hereinafter, example embodiments of the disclosure will be described asfollows with reference to the accompanying drawings.

FIG. 1 illustrates development of a camera provided with a displaypanel.

Referring to FIG. 1, a display panel and an imaging unit (e.g., camera)included in a mobile phone have been developed to improve a screen share(or a screen occupancy rate) of a display panel. For example, an earliermobile phone included a rectangular display panel and an image unit maybe disposed to be exposed on an external bezel, while in later mobilephones, a shape of a display has been changed such that an area of theexternal bezel is decreased, as shown in FIG. 1.

Recently, to further increase a screen share of a display panel, amethod of removing an external bezel and providing a camera by drillinga hole in the display panel in a pop-up form has been developed. Also, aproduct equipped with an under-display camera (UDC), which is disposedunder a display, has been developed as a method to maximize a screenshare of a display panel.

However, in an under-display camera, since light should enter a lens ofa camera through a display panel, transmittance of light may be lowered.Also, an opaque area having a predetermined pattern arrangement on adisplay panel may further lower transmittance of light entering thereto.Accordingly, performance of an under-display camera may be directlyrelated to quality of an obtained image, and the development of anunder-display camera needs to overcome the issues such as color shift,deterioration of resolution, and the like.

In the related art, a method of qualitatively evaluating performance ofa camera with the naked eye is used, which is not accurate. According toexample embodiments, a quantitative performance evaluation method may beused to improve performance of an under-display camera.

An improved under-display camera according to example embodiments may beapplied to mobile phones and also to electronic devices such as laptopsand televisions.

FIG. 2 is a block diagram illustrating a device for evaluatingperformance of an under-display camera according to an exampleembodiment.

Referring to FIG. 2, a device 1 for evaluating performance of anunder-display camera in an example embodiment may quantitativelyevaluate performance of the camera. The performance evaluation device 1may include a light emitting unit 10, an evaluation image acquisitionunit 20, a controller 30, a memory 40, and a reference image acquisitionunit 50.

The light emitting unit 10 may allow light to be incident to thereference image acquisition unit 50 and/or the evaluation imageacquisition unit 20 to generate a reference image and/or an evaluationimage. Performance of the evaluation image acquisition unit 20 may beevaluated according to an example embodiment based on the referenceimage and the evaluation image. Accordingly, the light emitting unit 10may allow light to be incident to the reference image acquisition unit50 and the evaluation image acquisition unit 20 under the sameconditions. However, an example embodiment is not limited thereto.

The evaluation image acquisition unit 20 may be configured to include anunder-display camera, and may include a display panel 22 and an imagingunit 25. For example, the display panel 22 and the imaging unit 25 maybe disposed in order in a direction in which light is incident. Lightincident from the light emitting unit 10 may pass through the displaypanel 22 and may be input to the imaging unit 25. The display panel 22may include an opaque area having a regular pattern, and the imagingunit 25 may generate an evaluation image having deteriorated imagequality due to the opaque area of the display panel 22. The generatedevaluation image may be converted or analyzed by the controller 30, anddata generated in this process may be stored in the memory 40.

The reference image acquisition unit 50 may be configured such that anunder-display camera is not included, and may include an imaging unit55. For example, light incident from the light emitting unit 10 may bedirectly incident to the imaging unit 55. The imaging unit 55 maygenerate a reference image by imaging incident light. Since thereference image is a comparison target to evaluate performance of theevaluation image acquisition unit 20 by analyzing the evaluation image,the imaging unit 55 included in the reference image acquisition unit 50may be configured to have the same performance as that of the imagingunit 25 included in the evaluation image acquisition unit 20.

The generated reference image may be converted or analyzed by thecontroller 30, and data generated in this process may be stored in thememory 40. A frequency of generating the reference image may not belimited to an example embodiment. As an example, performance of theevaluation image acquisition unit 20 may be evaluated based on aplurality of evaluation images using the reference image generated once.As another example, performance of the evaluation image acquisition unit20 may be evaluated by generating a reference image corresponding to theevaluation image whenever the evaluation image is generated.

The controller 30 may quantify information related to the evaluationimage and the reference image generated by the evaluation imageacquisition unit 20 and the reference image acquisition unit 50,respectively, and may compare the quantified information, therebyevaluating performance of the evaluation image acquisition unit 20. Forexample, a method of calculating a feature value by quantifyinginformation related to the evaluation image may correspond to a methodof calculating a reference value by quantifying information related tothe reference image. A process of calculating the feature value and thereference value will be described in greater detail later.

As an example, the controller 30 may quantify a degree to which an imageof an area having the same light intensity from each image is similar toa circular shape, and may compare the degrees with each other. Thecontroller 30 may output a performance evaluation result with respect tothe evaluation image acquisition unit 20 by comparing the feature valuerelated to the evaluation image with the reference value related to thereference image. For example, the controller 30 may store theperformance evaluation result of the evaluation image acquisition unit20 in the memory 40.

However, an example embodiment is not limited thereto. As an example,the performance evaluation device 1 may further include additionalelements if desired. Also, the controller 30 may further perform otherfunctions in addition to the described functions, and the memory 40 maystore at least a portion of data generated until the result ofevaluating performance of the evaluation image acquisition unit 20 isobtained after the evaluation image and the reference image aregenerated.

FIG. 3 is a flowchart illustrating a method of evaluating performance ofan under-display camera according to an example embodiment.

Referring to FIG. 3, performance of an under-display camera may beevaluated using a feature value quantified based on the evaluation imagegenerated according to an example embodiment.

Referring to FIGS. 2 and 3, light may be incident to the imaging unit 25through the display panel 22 (S110). The imaging unit 25 may generatethe evaluation image by imaging the incident light (S120).

As an example, since the evaluation image is obtained by imaging lightpassing through the display panel 22, image quality of the evaluationimage may be deteriorated as compared to that of a general image, whichis obtained by imaging light that has not passed through the displaypanel 22. The information of image quality deterioration may be includedin the evaluation image, and the controller 30 may calculate, based onthe evaluation image, a feature value, in which the information of imagequality deterioration is reflected (S130). Based on the feature value,the controller 30 may output a result of evaluating performance of theevaluation image acquisition unit 20 including the under-display camera(S140).

However, the operations of the method for evaluating the performanceillustrated in FIG. 3 are merely an example and an example embodiment isnot limited thereto. As an example, additional operations may be furtherincluded prior to, between, or after operations S110 to S140, or some ofoperations described in FIG. 3 may be omitted or modified.

FIGS. 4A and 4B are diagrams illustrating evaluating performance of anunder-display camera according to example embodiments.

FIG. 4A illustrates a process in which light L output by the lightemitting unit 10 is incident to the evaluation image acquisition unit20.

In an example embodiment, the light emitting unit 10 may include a lightsource 12 and a shielding box 15. For example, the light emitting unit10 may be configured as a point light source, the shielding box 15 mayinclude a slit, and a point light source may be disposed in theshielding box 15.

The slit of the shielding box 15 may have a predetermined width a. As anexample, the predetermined width a may be a value between 1 mm and 3 mm.However, an example embodiment is not limited thereto. For example, thepredetermined width a may have a value less than 1 mm or greater than 3mm.

The light emitting unit 10 may output light L that moves linearly in onedirection through the slit of the shielding box 15. The output light Lmay be incident to the evaluation image acquisition unit 20. As anexample, the evaluation image acquisition unit 20 may be configured toinclude an under-display camera. Accordingly, the incident light L maybe incident to the imaging unit 25 through the display panel 22. Forexample, the incident light L may be vertically incident to the displaypanel 22 and the imaging unit 25.

However, an example embodiment is not limited to the example illustratedin FIG. 4A, and the light emitting unit 10 configured to output thelight L that moves linearly in one direction may be implemented invarious forms.

FIG. 4B may be an example embodiment of the evaluation image acquisitionunit 20 in the form of a mobile phone. However, an example embodiment isnot limited thereto, and the form of the evaluation image acquisitionunit 20 may not be limited to the illustrated example.

As an example, the evaluation image acquisition unit 20 may include thedisplay panel 22 and the imaging unit 25. The display panel 22 maycorrespond to a screen of a mobile phone, and the imaging unit 25 maycorrespond to a front camera of the mobile phone. As an example, thedisplay panel 22 may be configured to completely cover the imaging unit25, and as described with reference to FIG. 4A, light incident to thedisplay panel 22 may pass through the display panel 22 and may beincident to the imaging unit 25. The imaging unit 25 may generate anevaluation image for evaluating performance of the evaluation imageacquisition unit 20 based on the incident light.

FIG. 5 is a diagram illustrating a display panel for applying a methodof evaluating performance of an under-display camera according to anexample embodiment.

Referring to FIG. 5, an evaluation image acquisition unit 20 forapplying a method of evaluating performance of an under-display camerain an example embodiment may include a display panel 22 and an imagingunit 25.

The display panel 22 may include a regular predetermined pattern,examples of which are illustrated in (a) to (f) in FIG. 5. Thepredetermined patterns may include an opaque area, and light incident tothe display panel 22 may cause a diffraction phenomenon due to thepredetermined patterns. Also, the opaque area included in thepredetermined patterns may reduce the amount of light incident to theimaging unit 25. For example, the amount of light incident to theimaging unit 25 may be further reduced in a range of about 50% to 90%than light incident to the display panel 22. Accordingly, the evaluationimage generated by the imaging unit 25 may have deteriorated quality.

A shape of the evaluation image generated in an example embodiment maybe determined based on a predetermined pattern included in the displaypanel 22. With respect to the method of evaluating performance of theunder-display camera in an example embodiment, since a performanceevaluation result may be obtained from the evaluation image, thepredetermined pattern included in the display panel 22 may directlyaffect the performance evaluation result.

Referring to pattern (a), the display panel 22 may include a patternhaving a regular circular hole therein. Referring to pattern (b), thedisplay panel 22 may include a pattern having regular oval-shaped holestherein. Also, referring to patterns (c) and (d), the display panel 22may include a pattern having holes of a polygonal shape, such ashexagons or octagons, therein.

However, the predetermined pattern included in the display panel 22 isnot limited thereto, and the display panel 22 may include a patternhaving a circular hole and polygonal holes surrounding the circularhole, as illustrated in pattern (e). Also, as illustrated in pattern(f), the display panel 22 may include a pattern having holes of the sameshape at positions arranged in one direction and holes of differentshapes at different positions in one direction. However, the patternsincluded in the display panel 22 are not limited to the examplesillustrated in (a) to (f), and the display panel 22 may include patternshaving various shapes.

FIG. 6 is a flowchart illustrating a method of evaluating performance ofan under-display camera according to an example embodiment.

FIG. 6 illustrates a method of calculating a reference value to evaluateperformance of an under-display camera according to an exampleembodiment.

Referring back to FIG. 2, the reference image acquisition unit 50 maygenerate a reference image by imaging light incident from the lightemitting unit 10 (S210). Since light incident to the reference imageacquisition unit 50 is directly incident to the imaging unit 55 withoutpassing through a display panel, the reference image may be a generalimage having less deterioration in image quality.

The controller 30 may convert the generated reference image into a grayscale image (e.g., black and white image) (S220), and may convert theconverted gray scale image into a contour image (S230). Based onoperation S220 of converting the reference image into the gray scaleimage to generate the contour image, accuracy of analysis of the contourimage may be improved. However, an example embodiment is not limitedthereto, and the reference image may be directly converted into acontour image without performing operation S220.

The controller 30 may calculate a reference value based on the generatedcontour image (S240). As an example, the reference value may be obtainedby quantifying a degree to which an image of an area having the samelight intensity in the reference image is similar to a circular shape.For example, since the reference image is a general image with lessdeterioration in image quality, an image of an area having the samelight intensity may appear similar to a circular shape. Accordingly, thereference value may include image information of when there is lessdeterioration in image quality.

FIG. 7 is a flowchart illustrating a method of evaluating performance ofan under-display camera according to an example embodiment.

FIG. 7 illustrates a method of calculating a feature value forevaluating performance of an under-display camera for comparison with apre-calculated reference value according to an example embodiment.

Referring back to FIG. 2, the imaging unit 25 included in the evaluationimage acquisition unit 20 may generate an evaluation image by imagingincident light passing through the display panel 22 (S310). As describedabove, the evaluation image may appear to have deteriorated quality.

Similarly to the method of converting the reference image illustrated inFIG. 6, the controller 30 may convert the generated evaluation imageinto a gray scale image (e.g., black and white image) (S320), and mayconvert the converted gray scale image into a contour image again.(S330). However, an example embodiment is not limited thereto, and theevaluation image may be directly converted into a contour image withoutperforming operation S320.

The controller 30 may calculate a feature value based on the generatedcontour image (S340). Similarly to the reference value, the featurevalue may be obtained by quantifying a degree to which an image of anarea having the same light intensity in the evaluation image is similarto a circular shape. Since the evaluation image has deteriorated imagequality, the feature value may include image information different fromthat of the reference value.

In an example embodiment, a result of evaluating performance of theevaluation image acquisition unit 20 may be calculated by comparing thefeature value with the reference value (S350). Accordingly, how similarthe evaluation image is to a circular shape and how much noise has beengenerated may be predicted.

FIGS. 8 and 9 are diagrams illustrating images generated using a methodof evaluating performance of an under-display camera according to anexample embodiment.

FIGS. 8 and 9 illustrate examples of the images described in theflowcharts in FIGS. 6 and 7. As an example, the images illustrated inFIG. 8 may be related to the reference image acquisition unitcorresponding to FIG. 6, and the images illustrated in FIG. 9 may berelated to the evaluation image acquisition unit corresponding to FIG.7.

Referring to FIG. 8, a reference image 110 generated by the referenceimage acquisition unit may be converted (810) to a gray scale image 120,and the gray scale image 120 may be converted (820) to a contour image130. For further detailed analysis, the contour image 130 may beconverted (830) into a three-dimensional (3D) contour image 140.However, an example embodiment is not limited thereto, and the aboveoperations may be changed to, for example, directly converting the grayscale image 120 to the 3D contour image 140 or directly converting thereference image 110 to the contour image 130.

As described above, since the images 110, 120, 130, and 140 related tothe reference image acquisition unit are based on the reference image110 with less deterioration in image quality, the shapes thereof may besimilar to a circular shape.

Referring to FIG. 9, an evaluation image 210 generated by the evaluationimage acquisition unit may be converted (910) to a gray scale image 220,and the gray scale image 220 may be converted (920) to a contour image230. For further detailed analysis, the contour image 230 may beconverted (930) into a 3D contour image 240. However, an exampleembodiment is not limited thereto, and the above operations may bechanged to, for example, directly converting the gray scale image 220 tothe 3D contour image 240 or directly converting the reference image 210to the contour image 230.

As described above, since the images 210, 220, 230, and 240 related tothe evaluation image acquisition unit are based on the evaluation image210, which have the deterioration of image quality, the shapes thereofmay not be a circular shape, and may have a different shape. Forexample, the shape of the images 210, 220, 230, and 240 may bedetermined based on a predetermined pattern of the display panelincluded in the evaluation image acquisition unit.

FIGS. 10 and 11 are diagrams illustrating a parameter used in analysisof an image with respect to a method of evaluating performance of anunder-display camera according to an example embodiment.

FIG. 10 illustrates an enlarged portion of the contour image 130 basedon the reference image 110 illustrated in FIG. 8, and FIG. 11illustrates an enlarged portion of the contour image 230 based on thereference image 210 illustrated in FIG. 9.

According to the method for evaluating performance of the under-displaycamera in an example embodiment, a reference value and a feature valuemay be calculated by quantifying a degree to which an area having thesame intensity of light is similar to a circular shape from thegenerated contour images 130 and 230.

Referring to FIG. 10, in each of a plurality of positions havingcoordinates of (x_(i), y_(i)) on the contour image 130 (or a boundary ofthe contour image 130), the reference value may be calculated based on aphase angle (θ_(i)) with respect to a center of the contour image 130having coordinates of (x_(C1), y_(C1)) and a distance Di from the centerof the contour image 130 to the plurality of positions.

Referring to FIG. 11, in each of the plurality of positions havingcoordinates of (x_(j), y_(j)) on the contour image 230 (or a boundary ofthe contour image 230), the feature value may be calculated based on aphase angle (θ_(j)) with respect to the center of the contour image 230having coordinates of (x_(C2), y_(C2)) and a distance Dj from the centerof the contour image 230 to the plurality of positions.

Parameters for calculating the reference value and the feature value maybe derived from Equations 1 and 2 as below.

$\begin{matrix}{{\theta_{i} = {{atan}\left( \frac{x_{i} - x_{C\; 1}}{y_{i} - y_{C\; 1}} \right)}}{\theta_{j} = {{atan}\left( \frac{x_{j} - x_{C\; 2}}{y_{j} - y_{C\; 2}} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack\end{matrix}$D _(i)=√{square root over ((x _(i) −x _(C1))²+(y _(i) −y _(C1))²)}D _(j)=√{square root over ((x _(j) −x _(C2))²+(y _(j) −y_(C2))²)}  [Equation 2]

The phase angle and the distance parameters derived from Equations 1 and2 may be used to indicate coordinates of the plurality of positions onthe contour images 130 and 230. Also, the parameters may be used tocalculate the reference value and the feature value.

FIGS. 12 and 13 are diagrams illustrating a method of calculating areference value and a feature value with respect to a method ofevaluating performance of an under-display camera according to anexample embodiment.

FIG. 12 may be a diagram illustrating each position on the contour image130 illustrated in FIG. 10 according to a distance and a phase angle,and FIG. 13 may be a diagram illustrating each position on the contourimage 230 illustrated in FIG. 11 according to a distance and a phaseangle. For example, it may be determined that the smaller thefluctuation in the graph illustrated in FIG. 13, the more the contourimage 230 converted from the evaluation image 210 may be similar to acircular shape.

Based on the above process, a reference value and a feature value may becalculated from FIGS. 12 and 13, respectively. For example, thereference value and the feature value may include at least one of anamplitude of diffraction, a standard deviation of diffraction, and anaverage of diffraction.

As an example, the amplitude of diffraction may be defined as adifference between a distance to a position in which the distance is themaximum and a distance to a position in which the distance is theminimum in FIGS. 12 and 13. The amplitude of diffraction may becalculated from Equation 3 as below.Amplitude_(diffraction) =D _(max) −D _(min)  [Equation 3]

The standard deviation of diffraction may be defined as a standarddeviation of the distance for overall phase angles in FIGS. 12 and 13.The standard deviation of diffraction may be calculated from Equation 4as below.

$\begin{matrix}{{STDEV}_{diffraction} = \sqrt{\sum\limits_{i,{j = 1}}^{n}{\left( {D_{i,j} - \overset{\_}{D}} \right)^{2} \times \frac{1}{n}}}} & \left\lbrack {{Equation}\mspace{14mu} 4} \right\rbrack\end{matrix}$

Also, the average of diffraction may be defined as an average of thedistances for overall phase angles in FIGS. 12 and 13. The average ofdiffraction may be calculated from Equation 5 as below.

$\begin{matrix}{{Mean}_{diffraction} = {{\sum\limits_{i,{j = 1}}^{n}{D_{i,j} \times \frac{1}{n}}} = \overset{\_}{D}}} & \left\lbrack {{Equation}\mspace{14mu} 5} \right\rbrack\end{matrix}$

The reference value and the feature value may be calculated using one ormore of Equations 3 to 5. With respect to the performance evaluationmethod in an example embodiment, a ratio of the reference value to thefeature value may be output as a performance evaluation result.Accordingly, it may be determined that the more the performanceevaluation result is approximate to 100%, the less the differencebetween the evaluation image and the reference image may be, and thebetter the performance of the evaluation image acquisition unit may be.

When each of the reference value and the feature value includes aplurality of values, there may be a plurality of performance evaluationresults. In this case, a score for the performance of the evaluationimage acquisition unit may be calculated based on the plurality ofperformance evaluation results. For example, the score for theperformance of the evaluation image acquisition unit may be an averagevalue of a plurality of performance evaluation results. However, anexample embodiment is not limited thereto, and a score for theperformance of the evaluation image acquisition unit may be calculatedby assigning a weight to each of the performance evaluation results.Scores calculated for a plurality of under-display cameras may becompared with each other, thereby determining an under-display camerahaving better performance.

FIG. 14 is a diagram illustrating a performance evaluation result of amethod of evaluating performance of an under-display camera according toan example embodiment.

The performance evaluation results and scores according to the method ofevaluating performance of the under-display camera in an exampleembodiment described with reference to FIGS. 1 to 13 may be obtained asin FIG. 14 as an example.

Referring to FIG. 14, the reference value calculated based on thereference image generated by the reference image acquisition unit may beX0, Y0, and Z0, and the feature value calculated based on the evaluationimage generated by the evaluation image acquisition unit may be X1, Y1,and Z1. For example, X, Y, and Z may correspond to an average ofdiffraction, a standard deviation of diffraction, and an amplitude ofdiffraction, respectively.

Based on the calculated reference value and the feature value, theresult of evaluating performance of the under-display camera in theexample embodiment may be obtained as 75%, 10%, and 15%, respectively.When the score is calculated from the average of the performanceevaluation results, the score may be determined to be 33.3%.

FIG. 15 is a flowchart illustrating a method of evaluating performanceof an under-display camera according to an example embodiment.

Referring to FIG. 15, the method for evaluating the performance of theunder-display camera in the example embodiment illustrated in FIGS. 1 to14 may further improve performance of the under-display camera by addinga feedback process.

Operations S410 to S460 of generating a reference image and anevaluation image, calculating a reference value and a feature value, andoutputting a performance evaluation result therefrom may correspond tothe operations illustrated in FIGS. 3, 6, and 7. For example, in anexample embodiment, light may be incident to an imaging unit through adisplay panel (S410), and an evaluation image may be generated byimaging the incident light (S420). Thereafter, the evaluation image maybe converted into a gray scale image (e.g., black and white image)(S430), the gray scale image may be converted into a contour image(S440), and a feature value may be calculated (S450). The referencevalue may be calculated by the same method except that light that doesnot pass through the display panel is imaged, and a performanceevaluation result for the evaluation image acquisition unit may beoutput based on the feature value and the reference value (S460).

The evaluation image generated in the above operation(s) may be a firstevaluation image, the calculated feature value may be a first featurevalue, and the performance evaluation result based on the first featurevalue may be a first performance evaluation result.

The first performance evaluation result may be output, and it may bedetermined whether to change an external condition and to performadditional performance evaluation (S470). As an example, the externalcondition may include presence of noise and intensity of noise of whenan image is generated by the imaging unit. As an example, noise mayrefer to external light, but an example embodiment is not limitedthereto. When additional imaging under changed conditions is necessary,an external condition for evaluating performance of the evaluation imageacquisition unit may be changed (S475).

Thereafter, a second evaluation image may be generated through the sameoperations S410 to S460, and a second feature value may be calculatedfrom the second evaluation image. A second performance evaluation resultmay be output by comparing a second feature value with the referencevalue. In this case, the reference value may have a value different froman existing reference value according to changes in external conditions,but an example embodiment is not limited thereto. By defining theexternal conditions evaluated as above, influence of noise may bereduced, and performance of the under-display camera having beenobjectively quantified may be evaluated.

With respect to the method for evaluating performance of theunder-display camera in an example embodiment, an optimization forimproving the performance of the under-display camera may be performedseparately from the operation S470. For example, a first performanceevaluation result may be output, and it may be determined whether theperformance of the evaluation image acquisition unit may be improved bycontrolling an evaluation device (S480). When performance improvement ispossible, the controller may control at least one of the light emittingunit, the display panel, and the imaging unit based on the firstperformance evaluation result before generating the second evaluationimage (S485).

Thereafter, a second evaluation image may be generated through the sameoperations S410 to S460, and a second feature value may be calculatedfrom the second evaluation image. An improved second performanceevaluation result may be output by comparing the second feature valuewith the reference value.

Although not illustrated in FIG. 15, with respect to the method ofevaluating performance of the under-display camera in an exampleembodiment, performance evaluation result prediction according toadjustment of the under-display camera may be performed separately fromthe operations S470 and S480. For example, even in a state in which theperformance evaluation device is not physically controlled as in S485,the second performance evaluation result according to changes in thelight emitting unit, the display panel, and the imaging unit may bepredicted based on the first performance evaluation result. As anexample, a performance evaluation result of when a regular patternincluded in the display panel has a different shape may be predicted.

From the performance evaluation result predicted according to the changein the performance evaluation device, the development direction of theunder-display camera may be easily determined. Also, the predictedperformance evaluation result may be used for diffraction correction toimprove performance of the under-display camera, and also forcalibration for maximizing performance during mass production.

When the first performance evaluation result and the second performanceevaluation result include a plurality of values, a first scorecalculated based on the first performance evaluation result, and asecond score calculated based on the second performance evaluationresult may be calculated. For example, the score for the performance ofthe evaluation image acquisition unit may be an average value of aplurality of performance evaluation results. However, an exampleembodiment is not limited thereto, and a score for performance of theevaluation image acquisition unit may be calculated by assigning weightsto each of the performance evaluation results. Whether to improve theperformance may be confirmed by comparing the scores calculatedaccording to the feedback operation of the under-display camera. As anexample, the second score based on the second performance evaluationresult in FIG. 15 may be greater than the first score based on the firstperformance evaluation result.

The output performance evaluation result may contribute to determining aprocess standard of each of elements included in an electronic deviceincluding an under-display camera. Also, a level of mass production of aproduct may be determined based on the output performance evaluationresult.

FIG. 16 is a diagram illustrating a performance evaluation result of amethod of evaluating performance of an under-display camera according toan example embodiment.

The performance evaluation results and scores according to the method ofevaluating performance of the under-display camera in an exampleembodiment described with reference to FIGS. 1 to 15 may be obtained asin FIG. 16 as an example.

For example, the performance evaluation may be sequentially performedunder a first condition in a dark state and a second condition in whicha certain amount of noise is present. Also, optimization of theevaluation device may be performed, and performance evaluation may beperformed sequentially under the first condition and the secondcondition again.

Under the first condition, the reference values calculated based on thereference image generated by the reference image acquisition unit may beX01, Y01, Z01, and first feature values calculated based on the firstevaluation image generated by the evaluation image acquisition unit maybe X11, Y11, and Z11. Under the second condition, the reference valuesmay be X02, Y02, and Z02, and the first feature values calculated basedon the first evaluation image may be X12, Y12, and Z12. For example, X,Y, and Z may correspond to an average of diffraction, a standarddeviation of diffraction, and an amplitude of diffraction, respectively.

Based on the reference value and the first feature value calculatedunder the first condition, the performance evaluation results of theunder-display camera in the example embodiment may be obtained as 75%,10%, and 15%, respectively. When a score is calculated by an average ofthe performance evaluation results, the score may be determined to be33.3%.

Based on the reference value and the first feature value calculatedunder the second condition, the performance evaluation results of theunder-display camera in the example embodiment may be obtained as 80%,15%, and 20%, respectively, and when a score is calculated by an averageof the performance evaluation results, the score may be determined to be38.3%.

As described above, when a certain amount of noise is present, theresult of evaluating performance of the under-display camera may have ahigher score. However, an example embodiment is not limited thereto. Asan example, each of the performance evaluation result values illustratedin FIG. 16 may be varied in example embodiments. Also, the result ofevaluating performance of the under-display camera may be a lower scoreunder the second condition in which noise is present.

After the performance evaluation based on the first feature value isterminated, an operation of controlling the evaluation device may beperformed to improve the performance of the evaluation image acquisitionunit. Accordingly, a second performance evaluation result different fromthe first performance evaluation result may be output.

The second feature value may be calculated under each of the first andsecond conditions, and may be compared with the reference value. Forexample, under the first condition, the second feature values calculatedbased on the second evaluation image generated by the evaluation imageacquisition unit may be X21, Y21, and Z21. Under the second condition,the second feature values calculated based on the second evaluationimage may be X22, Y22, and Z22.

Based on the reference value and the second feature value calculatedunder the first condition, the result of evaluating performance of theunder-display camera in the example embodiment may be obtained as 85%,30%, and 35%, respectively. When the score is calculated by an averageof the performance evaluation results, the score may be determined to be50%.

Based on the reference value and the second feature value calculatedunder the second condition, the results of evaluating performance of theunder-display camera in the example embodiment may be obtained as 90%,45%, and 50%, respectively, and when the score is calculated by anaverage of the performance evaluation results, the score may be 61.6%.

As described above, by undergoing the feedback operation for theunder-display camera, the performance evaluation result may have ahigher score. However, an example embodiment is not limited thereto. Asan example, each of the performance evaluation result values illustratedin FIG. 16 may be varied in example embodiments. Also, the result ofevaluating performance of the under-display camera may be a lower scoreunder the second condition in which noise is present.

According to the aforementioned example embodiments, the under-displaycamera may obtain the evaluation image using the evaluation imageacquisition unit including a UDC, and based thereon, a feature value forquantitatively evaluating the performance of the UDC, and a performanceevaluation result may be obtained. Accordingly, the UDC may becontrolled according to the performance evaluation result. For example,the development direction of a UDC may be determined, and optimizationto improve performance of the UDC may be performed.

At least one of the components, elements, modules or units describedherein may be embodied as various numbers of hardware, software and/orfirmware structures that execute respective functions described above,according to an example embodiment. For example, at least one of thesecomponents, elements or units may use a direct circuit structure, suchas a memory, a processor, a logic circuit, a look-up table, etc. thatmay execute the respective functions through controls of one or moremicroprocessors or other control apparatuses. Also, at least one ofthese components, elements or units may be embodied by a module, aprogram, or a part of code, which contains one or more executableinstructions for performing specified logic functions, and executed byone or more microprocessors or other control apparatuses. Also, at leastone of these components, elements or units may further include orimplemented by a processor such as a central processing unit (CPU) thatperforms the respective functions, a microprocessor, or the like. Two ormore of these components, elements or units may be combined into onesingle component, element or unit which performs all operations orfunctions of the combined two or more components, elements of units.Also, at least part of functions of at least one of these components,elements or units may be performed by another of these components,element or units. Further, although a bus is not illustrated in theblock diagrams, communication between the components, elements or unitsmay be performed through the bus. Functional aspects of the aboveexample embodiments may be implemented in algorithms that execute on oneor more processors. Furthermore, the components, elements or unitsrepresented by a block or processing operations may employ any number ofrelated art techniques for electronics configuration, signal processingand/or control, data processing and the like.

While some example embodiments have been illustrated and describedabove, it will be apparent to those skilled in the art thatmodifications and variations may be made without departing from thescope of the disclosure as defined by the appended claims and theirequivalents.

What is claimed is:
 1. An under-display camera disposed below a displayand configured to generate an image based on a light passing through thedisplay, wherein an image acquisition unit including the under-displaycamera is configured to generate an evaluation image based on a firstlight, by using an imaging unit including an image sensor, the firstlight being output by a light emitting unit, including a light source,and passing through a display panel, and wherein the under-displaycamera is controlled based on a result of evaluating a performance ofthe under-display camera, the result being obtained by calculating afeature value based on the evaluation image, calculating a referencevalue based on a reference image, and comparing the feature value withthe reference value, the reference image being obtained based on asecond light output by the light emitting unit, the second light notpassing the display panel.
 2. The under-display camera of claim 1,wherein the result of evaluating the performance is obtained based on aratio of ratio of the reference value to the feature value.
 3. Theunder-display camera of claim 1, wherein the result of evaluating theperformance is obtained by converting the evaluation image to a grayscale image and obtaining the feature value based on the gray scaleimage.
 4. The under-display camera of claim 1, wherein the result ofevaluating the performance is obtained by converting the evaluationimage to a contour image and obtaining the feature value based on thecontour image.
 5. The under-display camera of claim 1, wherein theresult of evaluating the performance by calculating the feature valuebased on a first degree to which an outline of the evaluation image issimilar to a circular shape.
 6. The under-display camera of claim 5,wherein the result of evaluating the performance is obtained bycalculating the reference value based on a second degree to which anoutline of the reference image is similar to the circular shape, andcomparing the first degree with the second degree.
 7. An under-displaycamera configured to, when disposed below a display, generate an imagebased on a light passing through the display, wherein an imageacquisition unit including the under-display camera is configured togenerate an evaluation image based on a first light by using an imagingunit including an image sensor, the first light being output by a lightemitting unit, including a light source, and passing through a displaypanel, and wherein the under-display camera is controlled based on aresult of evaluating a performance of the under-display camera, theresult being obtained by converting the evaluation image to a contourimage; calculating, in each of a plurality of positions on an outline ofthe contour image, a feature value based on a relative location of eachposition with respect to a center of the contour image; and comparingthe feature value with a reference value obtained from a referenceimage, the reference image being obtained based on a second light notpassing through the display panel.
 8. The under-display camera of claim7, wherein the wherein the contour image is obtained by converting theevaluation image to a gray scale image, converting the gray scale imageto the contour image.
 9. The under-display camera of claim 7, whereineach of the reference value and the feature value includes at least oneof an amplitude of diffraction, a standard deviation of diffraction, oran average of diffraction.
 10. The under-display camera of claim 9,wherein each of the reference value and the feature value includes theamplitude of diffraction, which is defined by a difference between amaximum distance to a first position on the outline of the contour imagefrom the center of the contour image and a minimum to a second positionon the outline of the contour image from the center of the contourimage, among the plurality of positions.
 11. The under-display camera ofclaim 9, wherein each of the reference value and the feature valueincludes the standard deviation of diffraction, which is defined as astandard deviation of a distance from the center of the contour image tothe plurality of positions.
 12. The under-display camera of claim 9,wherein each of the reference value and the feature value includes theaverage of diffraction, which is defined as an average of distances fromthe center of the contour image to the plurality of positions.
 13. Theunder-display camera of claim 7, wherein the relative location of eachposition is defined by a phase angle with respect to the center of thecontour image and a distance from the center of the contour image. 14.An under-display camera disposed below a display and configured togenerate an image based on a light passing through the display, whereinan image acquisition unit including the under-display camera isconfigured to generate a first evaluation image and a second evaluationimage based on a first light, by using an imaging unit including animage sensor, the first light being output by a light emitting unit,including a light source, and passing through a display panel, andwherein the under-display camera is controlled based on a result ofevaluating a performance of the under-display camera, the result beingobtained by calculating a first feature value based on the firstevaluation image and calculating a second feature value based on thesecond evaluation image; and obtaining a first performance evaluationresult by comparing the first feature value with a reference valueobtained from a reference image and obtaining a second performanceevaluation result by comparing the second feature value with thereference value, the reference image being obtained based on a secondlight not passing through the display panel.
 15. The under-displaycamera of claim 14, wherein the image acquisition unit is furtherconfigured to generate the first evaluation image and the secondevaluation image under different external conditions.
 16. Theunder-display camera of claim 15, wherein the different externalconditions relate to a presence of a noise and an intensity of the noisewith respect to the first evaluation image or the second evaluationimage.
 17. The under-display camera of claim 14, wherein the imagingunit of the image acquisition unit is controlled based on the firstperformance evaluation result prior to generation of the secondevaluation image.
 18. The under-display camera of claim 14, wherein,based on the first performance evaluation result and the secondperformance evaluation result respectively including a plurality ofvalues, the performance of the under-display camera is evaluated bycalculating a first score from the plurality of values of the firstperformance evaluation result and a second score from the plurality ofvalues of the second performance evaluation result.
 19. Theunder-display camera of claim 14, wherein a diffraction correction isperformed on the under-display camera based on the result of evaluatingthe performance.
 20. The under-display camera of claim 14, wherein theunder-display camera is controlled based on the first performanceevaluation result such that, after the controlling, the imageacquisition unit is configured to generate the second evaluation imageof which the second feature value is closer to the reference value.